Technology of capturing real space using a capture device mounted on a moving object, and representing the captured real space as virtual space using a computer based on the captured image data is proposed (for example, ‘Building Image-Based Cybercities by Using Vehicle-Mounted Cameras’ by Endo, Katayama, Tamura, Hirose, Watanabe, and Tanigawa (published by Shingaku Society, PA-3-4, pp. 276–277, in 1997), ‘Building Image-Based Cybercities by Using Vehicle-Mounted Cameras(2)—Generation of Wide-Range Virtual Environment by Using Photorealistic Images —’ by Hirose, Watanabe, Tanigawa, Endo, Katayama, and Tamura (published in Proceedings of the Virtual Reality Society of Japan Second Annual Conference, pp. 67–70, in 1997), etc.
A method for representing real space captured as virtual space based on the captured image data captured by the capture device mounted on a moving object can be a method of reconstructing a geometric model of real space based on captured image data, and representing it in the conventional CG technology. However, the technology is limited in correctness, precision, reality, etc. of a model. On the other hand, the image-based rendering (IBR) technology of representing virtual space using a captured image without using a model has recently attracted attention. The IBR technology produces an image from any viewpoint based on a plurality of captured images. Since the IBR technology uses captured images, realistic virtual space can be represented.
To produce walk-through system of virtual space using the above mentioned IBR technology, it is necessary to produce and present an image depending on the position in the virtual space of a user. In this type of system, each frame of captured image data is associated and stored with the position in the virtual space, the corresponding frame is obtained based on the position and the view direction of the user in the virtual space, thereby reproducing the frames.
As a common method of obtaining the position data in the real space, the position measurement system using an artificial satellite represented by the GPS (Global Positioning System) which is also used for a car navigation system, etc. can be used. A method of associating the position data obtained in the GPS with the captured image data can use a time code (Japanese Patent Publication Laid-open No. 11-168754). In this method, each frame of captured image data can be associated with position data by associating the time data contained in the position data with the time code added to each frame of captured image data.
In the walk-through presentation in the above mentioned virtual space, a user can see a desired direction from his or her viewpoint. Therefore, an image at each viewpoint is stored as a panoramic image covering a range wider than an image to be reproduced, a partial image to be reproduced is cropped and displayed from the panoramic image based on the position and the view direction of the user in the virtual space.
However, when a real city's view is captured by a capture device mounted on a vehicle as a moving object, image data of the same physical point is captured for a plurality of frames due to the condition of a traffic signal, the traffic jam, etc. Therefore, collected captured image data can contain a larger number of panoramic images at the same point. This apparently causes data overlapping. For example, wasteful panoramic images can be stored when a database of panoramic images are generated to represent virtual space. Furthermore, this also indicates wasting resources of storage media such as a hard disk, etc.